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ABSTRACT 

A programming language for natural language processing 
programs is described* Examples of the output of programs 
written using it are given- The reasons for various design 
decisions are discussed. An actual session with the system 
is presented, in which a small fragment of an English-to- 
French translator is developed. Some of the limitations of 
the system are discussed, along with plans for further 
development. 



A Linguistics Oriented Programming Language 
Vaughan\Pratt 

1. Overview 

This paper presents some aspects of work done at odd 
intervals over the past two years, first at Stanford and 
then at MIT, on a project to develop a programming language 
suitable for writing natural language processing programs* 
The relevant acronym is LINGOL, for Linguistics Oriented 
Language. Similar projects such as CGMIT (Tfngve 1963)# 
and its successors METEOR (Bobrow 19&* ) and SNOBOL (Farber 196*0 
no longer reflect the state of the art of computational 
linguistics; indeed, they do not rise above the remark that 
computational linguistics is concerned with processing text 
strings. The Issue addressed in these pages Is that of the 
programming technology appropriate to the syntax-semantics 
interface, an artifice that arises in the phrase-structure 
paradigm for natural languages. A secondary issue, to be 
dealt with elsewhere, concerns the relative merits of various 
parsing strategies for phrase-structure oriented grammars, 
and the development of a parsing algorithm superior to both 
the Earley and Cocke-Kasaral-Younger procedures- (See Aho 

and Ullman (1972), p. 31*0. 

Following Wlnograd's (1971) lead, we begin by giving 
some examples of the output of programs written in LINOOL. 
The point of having a programming language is to make 
programming less painful for all concerned. The interesting 
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property of these programs Is that two of them were written 
In quite a short space of time by students with no experience 
in either LINGOL or linguistics. Another program (the French 
translator) was designed, written and debugged from scratch 
for demonstration purposes by the author from 3;00 a.m. to 
3:00 a.m. of the morning he was scheduled to give a talk on 
it. 

The first program was written in September 1970, to test 
out the first version of LINQOL. It was a sort of "deep 
structure" analyzer which attempted to make syntactic remarks 
about sentences (Figure l). The grammar used in it served 
as the basis for the next two programs. 

The system languished for six months until a graduate 

student. Bill Faught, took advantage of it for a project in 
an A.I* workshop. He took two weeks to write an English- 
to-German translator (Figure 2). 

Later, Faught decided to do some serious work on question- 
answering systems, and soon produced a comprehension program 
(Figure 3) that relied on a relational model of the world in 
which all related concepts were represented in a graph as 
vertices linked by two-way, labelled edges. Recently he has 
produced considerably more impressive results, but it is more 
appropriate that Faught himself report on them. 

The French translator (Figure**) was written by the author 
early In 1972, for demonstration purposes. The program 
consisted of a page of grammar and semantics, a page of 
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NONE DF MY FRIENDS WERE EATEN BY A LI CM . 
ASSERTION: 

FALSE 

flCTDRl LIDM 
ACT: EAT 
DEJECT: FRIENDS 

PORTION: SOME 
OWNER: I 

THE AUTHORS OF NONE OF THESE BOOKS AF.E FRIENDS OF PETER 
ASSERTION: 

FALSE 

SUBJECT: AUTHORS 

SPECIFY: THE 
AUTHORS OF: BDOKS 

PORT I DM: SOME 
PLACE: HERE 
BE: FRIENDS 

FRIENDS DF: PETER 

IF A MAN CAN ERT A DOS A HORSE CAN EAT A SMALL CAT . 
ASSERTION: 

ACTOR: HORSE 
ACT: EAT 
OBJECT: CAT 

SMALLNESS: SMALL 
IMMEDIACY! POTENTIAL 
CONDITION: 

ACTOP: MAN 
ACT: EAT 
OBJECT: DOG 
IMMEDIACY: POTENTIAL 

HOW QUICKLY CAN YOU SUIM '. 
QUESTION! 

SPEED: 

ACTOR: YDU 

ACT: SUIM 

IMMEDIACY: POTENTIAL 

EAT A HORSE . 
COMMAND: 



ACTDP: YOU 
ACT: EAT 
OBJECT: HDRSE 

DD YDU LIKE ICECREAM . 
QUEST I DM: 

ACTDP: YDU 
ACT: LIKE 
DBJECT: ICECREAM 

Figure 1. "Deep Structure" Axalyser. 



!HE SLEEPS WITH HIM BECAUSE 'HE LOVES HIS CAT. 
SIE SCHLAEFT MIT IHM WEIL SIE SEINE KAT2E LIEET 

HE IS IH THE HOUSE. 
ER 1ST IN DEM HflUS 

HE HITS THE BfiLL TD THE DOG . 

ER 3CHLAEGT DAS BALL 2U DEM HUMD 

THE CRT IS IH THE TREE BY THE HOUSE. 
DIE KfiTZE 1ST IN DEM BfiUM NEBEN DEM HfiUS 

THE STUDENT WILL SLEEP BETTER WHEN HE UNDERSTANDS THE IDEA. 
DER STUDENT WIRD BESSER SCHLAFEN UENN ER DIE IDEE VERSTEHT 

THE LITTLE OLD MAN LOVES THE RED HOUSE BECRUSE HE CfiN SEE THE HORSES. 

DER KLEINE ALTE MANN LIEBT DAS RDTE HfiUS WEIL ER DIE PFERDE 
SEHEN KANN 

* 

WHILE THE FLOWER IS DLD THE APPLE IS YOUNG. ■ 
WAHREND DIE BLUME ALT 1ST 1ST DER APFEL JUNG 

SHE WILL GIVE HIM A FLOWER IF HE EATS THE APPLE TODAY. 

■ 

SIE WIRD IHM EINE BLUME GEBEN WENN EP DEN APFEL HEUTE ISST 

THE HORSE THAT EATS BAD APPLES IS A SICK HORSE. 

DAS PFERD DAS SCHLECHTE AEPFEL ISST 1ST EIN KRfiMKES PFEPD 

IF A HORSE CAN EAT A DOG A MAN CAN EAT A SMALL CAT. 

WENN EIN PFERD EINEN HUND ESSEN KANN KANN EIN MANN EINE 
KLEINE KATZE ESSEN 



Figure 2. The German Translator of W. Faught. 
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JACK BUILT ft PED HOUSE. HE LOVED PATS. MORTIMER WAS JACK '3 
PAT. HE LIVED IN THE HOUSE THAT JACK BUILT. 
- JILL BUILT A ELUE HOUSE. CHE LOVED CATS. SPOT WAS ft CAT WHQ 

..TE MORTIMER. JACK HATED JILL BECAUSE SPOT ATE MOPTIMEP. 

FIDD WAS A BROWN DOG. HE CHASED SPOT. SPIKE WHS A BLACK DO* . 
HE CHASED JILL. 

WHAT IS THE COLDP OF THE DO'S THAT CHASED THE CAT. 
BROUN 

DID JACK LOVE MORTIMER. 
YES 

WHO ATE MORTIMER. 
SPOT 

WHAT IS THE COLOR OF THE HOUSE IN WHICH THE PAT THAT THE CAT ATE LIVED. 

WHY DID JACK HATE JILL. 
BECAUSE SPOT EAT MORTIMER 

WHO DID THE BLACK DOG CHASE, 
JILL 

DID JILL LOVE ANY ANIMALS. 
YES 

iMEN HATE PATS. DID JILL DISLIKE MDRTIMER. 

Figure 3. Faught' s Question-Answering System. 

I LOVE MY WIFE. 
J' AIME MA FEMME 

SHE LOVES ME. 
ELLE M' RIME 

WE HAVE SOME BIG PED DOGS. 

NOUS AVCNS QUELQUES GRANDS CHIENS ROUGES 

WE LOVE OUR DOGS. 
NDUS AIMONS NOS CHIENS 

DUR DOGS LOVE US . . 
MOS CHIENS NOUS AIMENT 

BIG DOGS LOVE BEAUTIFUL PED TABLES. 

LES GRANDS CHIENS AIMENT LES BELLES TABLES ROUGES 

SMALL DOGS LIKE PRETTY LITTLE TABLES. 

»-FS PETITS CHIENS AIMEhT LES'JDLIES PETITES TABLES 

Figure i*. The.5-Hour French Translator. 
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actionary and a page of useful functions for conjugating 
verbs, arranging agreements of articles and adjectives, 
performing liason and so on, so it was not particularly large. 
The point of it was 
(i) it was easy to write; 

(11) it was sufficiently succinct to be suitable for exhibiticn; 
and 

(Hi) it dealt competently with that part of English for which 
it was defined. 

It is easy to claim that, since this is a toy translator, 
it says nothing about the real world. This is certainly true 
with respect to polysemy. However, it is false with respect 
to extensibility of grammatical rules; we shall later demon- 
strate the striking effects obtained on adding very simple 
rules. More recently, another four hours of work gave a self- 
tutoring capacity to the program (Figure 5)- Notice how 
unknown words are correctly classified as to part of speech 
before the program requests information. 

We have been basking in these examples somewhat vicariously. 
It is very much like explaining the advantages and disadvan- 
tages of FORTRAN by exhibiting the output of some FORTRAN 
programs. Thus the reader should only infer from these 
examples the existence of LINGOL, and a lower bound on what 
can be achieved with it; he should infer its quality or lack 
of it not from here but from the following. 



THE FDLKEMAN GIVES THEM A TICIET. 

WHAT IS "THE ';DP A> POLICEMAN"" 

LE GENDARME 

WHAT IS "TO GIVE"? 

DQNMER 

WHAT IS "THE <0P fly TICKET"? 

LE BILLET 

LE GENDARME LEUR DGNHE UN BILLET. 

1*45. MILLISECONDS. 

THE LITTLE GIRLS WANT A RED PIG. 

WHAT IS "THE COR fl) GIRL"? 

LA FILLE 

WHAT IS "TO WANT"? 

VOULDIP 

CONJUGATE IT 

VEUX VEUX VEUT VOULDNS VOULEZ VEULEHT 

WHAT IS "THE CDR A> PIG"? 

LE CDCHON 

LES PETITES FILLES VEULEHT UH COCHON ROUGE. 

1561. MILLISECONDS. 

PIGS IS PIGS. 

LES CDCHONS SONT DES COCHONS. 
773. MILLISECONDS. 

I HAVE THE PEN DF MY AUNT. 

WHAT IS "THE COR fl> PEN"? 

LA PLUME 

WHAT IS "THE <COR fl> AUNT"? 

TANTE 

WHAT IS ITS GENDER <MASC DR FEM>? 

FEM 

J' AI LA PLUME DE MA TANTE. 

1157. MILLISECONDS. 

I WANT THE BUREAU DF MY ONCPUIT 

• 

(HELLD> 

TYPE SENTENCE FOLLOWED BY . ! OR ? 

I WANT THE BUREAU OF MY UNCLE. 

WHAT IS "THE <OR. fl> BUREAU"? 

LE BUREAU 

WHAT IS "THE <!OP fi> UNCLE"? 

L'ONCLE 

WHAT IS ITS GENDER <MflSC OR FEM>? 

M.-i : c 

JE VEUX LE BUREAU DE MDN ONCLE . 
IZ2Z. MILLISECONDS. 



Figure 5- The 9-Hour French Translator, in Ignorance irode. 



2* Terminology and Perspective 

Let us set the stage preparatory to giving some definitions- 
We need £ paradigm for computational linguistics progrems, end 
we choose the translation paradigm as best describing the 
LINGOL system. The translation paradigm characterizes natural 
language processing programs as translators from the natural 
source language to some natural or formal target language, 
whether French, LISP, structural descriptions, predicate 
calculus, conceptual dependency diagrams or what have you. 
No loss of generality is entailed here, for by simply making 
the target language a programming language, any other paradigm 
may be conveniently emulated* The obvious competitor is the 
stimulus - response paradigm, in which the input is seen as a 
stimulus that elicits an action* Again no loss of generality 
can occur, since a possible action is to emit an utterance. 
The main advocate of this paradigm is Narasimhan (1969)* 
although it appears to be the implicit paradigm in many extant 
programs. We prefer the former paradigm for no very good 
reason, although we do find it easier conceptually to manipulate 
and characterise utterances rather than actions. In particular, 
in the programming methodology to be described, large items 
are gradually built up from smaller ones, and it Is tricky to 
cast this in a stimulus -response format. 

Within the translation paradigm we shall identify two 
main phases, cognitive and generative* The cognitive phase is 
parsing. In which the Input is preprocessed until it is in a 
form convenient for operation on by the generative phase. 
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whlch then produces the translation as output. The paradigm 
itself does not require that one phase run to completion 
before the other can start. Indeed, Winograd's (1971) program 
makes effective use of feedback froir the partial results of 
his generative routines in guiding the cognitive routines, by 
attempting to build a semantic structure for, say, a Noun 
Group, before continuing with the parsing* 

We are now prep 3 red for the definitions- By syntax is 
meant all aspects of the source language involved In the 
cognitive phase, including such things as phrase structure 
rules and semantic markers- By semantics we refer to what is 
involved In going from the source language (after the syntactic 
preprocessing) to the target language. By pragmatics we mean 
knowledge about the universe of discourse, and the local 
context, that may be consulted by both the cognitive and 
generative phases as they make decisions. 

Each of these three concepts has teen used many times in 
the literature, with varying shades of meaning and precision, 
so we are not redefining previously well-defined terms. 
Rather, we see three main aspects to the programs written in 
LINGOL, and found three reasonably uncommitted terms with 
which to label them. (The first two definitions coincide more 
or less with those of Winograd (1971), so we are not too far 
afield.) 

(it may seem paradoxical to Include semantic markers in 
syntax, but this is Just the consequence of our usage of the 
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word semantics as opposed to that of t say, Katz and Fodor 
(196tt) . With respect to our usage, semantic markers represent 
an attempt to encode a tiny fragment of pragmatics into syntax 
(or into linguistics, to use the Katz and Fodor terminology, 
and their equation SEMANTICS = LINGUISTICS - SYNTAX). We do not 
want to make value Judgments about such an encoding; the 
example simply serves to illustrate the perspective induced 
by our definition.) 
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3- Design Philosophy 

There Is not one philosophy in LINQOL, but three, each 
tuned to the requirements of the three concepts defined above. 
In the current version of LINGOL, the philosophies are 
roughly as follows. 
3.1- Syntax 

Although this paper Is concerned mainly with the semantic 
component of LINGOL, It behoves us to consider syntax since 
the cognitive phase's output is the generative phase's Input. 
The central decision to be made here is the choice of 
representation for this output. It seems to be necessary to 
discover the relations between the words of the sentence, or 
the phrases of the sentence, or the entities denoted by those 
words or phraaea. Corresponding to each of these possibilities 
are dependency structures (Kays 196*, Simmons 19&0* phrase 
structures (almost everybody) and conceptual dependency 
networks (ochank 1970). Actually the first two are not 
mutually exclusive, since it is perfectly reasonable to 
construct structures which contain all the information of both 
techniques. We shall use the term syntactic structure to refer 
to such a coalition, to distinguish It from a concept structure , 

LINGOL is meant to be a practical system suitable for 
export and immediate use by practising computational linguists. 
The technology for phrase structure is far advanced over any 
other technology, and every successful program for the past 
eight yeara or so has unashamedly used it. Also, it is fairly 



-12- 



easy to convert a phrase structure system to a syntactic 
structure system, by tagging each phrase with the correspond!^ 
governing word together with pointers to the dependent phrases 
(and hence words). 

For these reasons, the decision was made to use phrase 
structure as the output of the cognitive phase, leaving the 
other representations as projects to be experimented with in 
the future. It is worth noting at this point that the ides 
of a concept structure is a very powerful one, especially in 
combination with Fillmore's (1968) notion of case, as suggested 
by Shank (1970). The notion of phrase concatenation is nowhere 
near as rich as that of case-based relations between concepts. 
On the other hand, this does not make phrase-structure a 
hopeless loser; in principle it is possible to construct 
these relations during the generative phase- However, Shank's 
point is that the information so discovered is vital to the 
cognitive phase. More recent phrase-structure systems, including 
those of Bobrow and Frazer (1969), Woods (1969), Winograd (1971) 
and the system described here make provision for discovering 
this sort of information while building the phrase structure. 
This immediately raises the question, why not build the 
concept structure anyway, since this information is being 
discovered? This point seems unanswerable, and is an excellent 
area for more research. In the cfse of LINOOL, we have a 
half-answer, in that we have developed what we feel is very 
nice programmingmethodology for dealing with phrase structure? 
during the generative phrase. An avenue for research is 
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to see if this methodology carries over to concept structures* 
Given that LXHGOL is based on phrase structure, the next 
Issue is that of the user's language for describing how that 
phrase-structure is to be built. The two criteria here are 
expressive power and ease of use- For our first iteration of 
LINQOLj since we were more interested in rapidly developing 
the semantics technology, we opted to sacrifice expressive 
power for ease of use if necessary. This corresponds in a 
way to Woods (1963) and Charnia^ (197?) assuming the existence 
of some sort of parser and continuing from there. The 
differences are firstly that both addressed pragmatic issues 
while we address semantic, and secondly that whereas they made 
up their own parsed output, LINGOL is equipped with a parser, 
on the philosophy that it is easier to type unparsed than 
parsed sentences, and that no harm is done when the parser 
gangs agley, which in practice occurs satisfactorily 

infrequently anyway. 

The user's language for the cognitive component was 
therefore chosen to he context-free rules, since these are 
very easy to write* They have exactly the same expressive 
capacity as Woods' (19^9) transition networks. Moreover, 
Just as Woods extended the capacity of these networks by 
allowing the user to specify operations on registers, so do 
we permit the user to supply code to give hints to the parser 
whenever it is about to apply a rule. This code has access 
to the part of the tree built so far by the parser and 
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relevant to the rule in question, and also to the user's 
data base, or pragmatics {which seems to ma^e semantic markers 
unnecessary as a special feature of LINGOL). The form of 
the hint is a grunt of approval or disapproval , at a volume 
appropriate for the particular hint, and in this repect is 
Just like Winograd's (1973) numerical treatment of ambiguity. 
So far, however, none of the programs written in LINGOL have 
made more th3n trivial use of this feature, in sharp contrast 
to the use made of the features in the semantics stage. 
With respect to the actual parser used, the syntax 
philosophy is that the parser should be transparent to the 
user, to within the representation cf the parts of the tree 
to which the user's code has access during the cognitive phase. 
This philosophy has enabled us to run without alteration each 
of a number of different LIHQOL programs in conjunction with 
various parsing algorithms. The details of these parsers 
and experiments are beyond the scope of this paper. 

3.2. Semantics 

In programming his semantics, the user should be able to 
work without the distracting detail of parsing, tree represen- 
tation, and ambiguity* The point of Identifying the cognitive 
and generative phases is to isolate these Issues logically in 
order to achieve this division of labor. Whether writing an 
English-toPrench translation program or a question-answering 
system, there are many details to worry about that have abso- 
lutely no relevance to the cognitive phase; the myriad 
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ldlosyncrasles of French grammar and style, the various 
searching algorithms and Inference rules that are tightly 
coupled in a QA system to the surface Structure information, 
and so on- Without some method in this large-scale madness, 
progress is bound to be slow- 
Furthermore, we believe that a high level of performance 
will be forthcoming from the cognitive phase of, say, machine 
translation programs, long before a similarly impressive level 
is attained by the generative phase. This is partly because 
comparatively little work is being done on generative aspects 
of MT, but more because it is inherently harder to say some- 
thing with good grammar and style than it Is simply to under- 
stand what is being said (at least explicitly!)* The 
cognitive phaae can ignore most details of style, and 

many details of grammar. In every program written so far with 
LINGOL, the generative component has been about three times 
the size of the cognitive component, and our prediction Is 
that this ratio will increase as each phase is improved. 

In taking this point of view, we are following a 
different philosophy from that of Wlnograd (1971), who makes 
use of strong interaction between the syntax and semantics 
components, which is one of the more notable features of his 
program. However, the result has been to produce a program 
whose details are lost In the rlchnecs of this interaction, 
and I have heard Winogred mutter when looking at a part of 
the program for M BE", "I don't remember writing that"* 
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Por the moment we are willing to sacrifice whatever 
additional power this approach has to offer for the sake of 
being able to write clean, modular, transparent semantic code. 
However, re do not believe that in order to restore this power 
we need to restore this interaction* Instead, we plan to rely 
eventually on strong interaction between syntax and pragmatics, 
leaving semantics as the cognition-independent arena. This is 
not Just passing the buck; since we see semantics as being more 
complex than syntax, we are trying to divide the work-load 
more evenly to keep all modules reasonable small. How syntax 
is to consult pragmatics is material for future research. Our 
point is that the bulk of semantics is irrelevant to syntax. 

The issue now is simply, how does one write programs 

that operate on trees (the output of LINQCL's cognitive 
phase)? This issue has been addressed by computer scientists 
in connection with compiling for the past ten years, and the 
discipline of syntax directed translation has gradually 
emerged. An early syntax directed translator is that of 
Warshall and Shapiro (1964). They used the tree-walk paradigm, 
in which the semantics consists of programs chat tell a pointer 
to move up, down or across the tree and occasionally output 
Information. Floyd (conversation) has commented that the 
technique was much too clumsy for practical applications when 
compared with techniques that tied the semantics to the syntax 
rather than to the output of the syntax. It is alarming to 
find Winograd using this approach in his program, which we 

conjecture would be made more transparent by adopting a more 
rule-oriented and less tree-oriented approach. 
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Some theoretical work has been done on syntax-directed 
translation, notably by Lewis and Stearns (1969), Knuth (1968), 
and Aho and Ullman (1972)- Knuth's paper Is of Interest In 
that It deals with the problem of passing information up and 
down a tree, using the notions of inherited (from above) and 
synthesized (from below) attributes. All of these studies 
suffer, from the computational linguist's point of view, in 
that they deal with the microcosm of computer source and target 
languages, in which the former can be made a compromise between 
the user's needs and the syntax-directed technology, and the 
latter Is a relatively well-defined, reference-poor language 
when compared with, say, French. 

Knuth's inherited and synthesized attributes come closest 
to meeting our needs* The problem with these attributes lies 
with his mechanism for moving them around a tree. Every node 
through which information is passed must make explicit provision 
for forwarding it, even if it is irrelevant to that node* 
For example, consider: 

No mother of such twins has time to relax- 
The mother of no such twins has time to relax* 
The mother of such twins does not have time to relax. 
The mother of such twins has no time to relax. 
(The second sentence is inspired by a study of negation by 
Klima (1964). It should be said in a tone of horror, with the 
emphasis on "no", before it sounds correct.) 
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In each case, what is being negated Is the whole sentence, 
yet the negation marker can be almost enyvhere in the sentence. 
This implies that a large number of rules will have to make 
provision for passing up a negation marker. 

This problem can be circumvented by using global variables 
instead of Knuth's attributes. Now all that is needed is for 
the negation marker to set a negation variable, and for the 
semantics at the syntactic clause level to read it. 

However, consider the following: 

The mother who has no twins has time to relax. 

This sentence makes a positive claim (as distinct from 
the negative one of the previous example) in that it says that 
there actually are people who do have time to relax, namely 
those mothers who have no twins. (Moreover, It does not 
explicitly say what happens to mothers of twins.) This seems 
to be a situation where synthesized attributes outperform 
global variables, since the rule at the relative clause level 
can simply refuse to pass on the negation marker. 

Negation is not the only such troublemaker. Arranging 
subject-verb, adjective-noun and determiner-noun agreement 
also requires passing information around the tree, especially 
when translating into French, where word-for-word translation 
does not necessarily result in correct agreement. Again, having 
more than one clause makes difficult the use of global 
variables, particular when a plural relative clause is separating 
a singular subject from its verb. Consider the five 
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subject-verb agreements in: 

As I walked into the saloon, the three men whom 

Jim talked to after I left him yesterday got up 

and slowly walked towards raa. 

All of these problems are "marker 11 type problems. Even 
worse is passing stylistic information from a word at the 
bottom of a tree to a clause node higher up, where this 
information is to be used to alter the whole structure of the 
translated clause. Again it ia important that the appropriate 
clause get this information. 

The mechanism we want here is that of the local variable , 
whose scope is the clause with which it is associated. With 
many clauses we will associate many more local variables 
corresponding to the various markers and other messages that 
each clause may want. Similarly, we will associate other 
local variables with noun phrases, to achieve adjective-noun 
and determiner-noun agreement. In the case of the subject, 
some of these markers (person and number, but not gender) 
must be shared with the clause as well, to ensure subject- 
verb agreement, but we do not want the clause to share the 
object's variables. Also, a relative clause such as "who 
sleeps" needs the same information from its govenor as does 
the principal clause. Moreover, we will want to pass not only 
markers, but also word-specific programs rritten at the 
dictionary level but intended for use at the clause or other 
level, (Winograd makes use of this technique for putting 
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the right programs in the right places-) The Implementation 
of local variables must be able to h*ndle these combinations. 

The first version of LINOOL implemented all of this in 
an unimaginative and not very general way. Eventually, we 
saw the light and came up with the program paradigm for 
syntax-directed translation. 

The program paradigm says that the surface structure 
tree 13 a program* At each node of the tree there is a 
function, and the subtrees of that node are the arguments of 
that function. For example, if we have a tree labelled 



print 
x 



t b c 



this corresponds to the program "print (a + b)x( (-c)-d)". 

Since LISP has a mechanism for local variables (two, in 
fact - PROO variables and LAMBDA variables), by adopting 
the program paradigm we automatically get local variables. 
Moreover, because we can write the code for each function 
separately, we attain a very high level of modularity, which 
we have found pays off handsomely when one tries to add new 
rules to an already operational LINGOL program. 
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The mechanism we use for running these programs differs 
slightly from LISP'S usual SVAL operator. The main difference 
is that it evaluates the function at each node first, giving 
the function the responsibility for evaluating subtrees at 
its leisure, and controlling the scopes of variable? for 
different subtrees. 

To illustrate all of this, we shall develop a small 
French translator. Imagine we are seated at a computer 
console. The following session has all typing errors and 
stupid mi3takes edited out, since they rapidly become boring 
and obscure the main issues* 

First we load the system- 

LtK! 

— * 

LISP 229CX 
ALL DC? 
PEADY-TO-PEAD-GPAMNAR : 

We are now talking to LISP. To get to talk to LIHGOL, 

type (HELLO). 

(HELLO) 

FIRST ENTER YOUR GRAMMAR . THEN SAY HELLO AGAIN. 

TOP-LEVEL: ..._ 



So LINGOL la not yet educated. We could tell LISP to 
read In our dictionary 3nd grammar from a file, but since we 
don't have such a file we will simply type it in at the 
terminal. 

First, let us give LISP a few words. 

(DICTIONARY) 
(THE DET '<LE)> 
(DOG NOUN '(CHIEN)) 
(SEA NOUN '(MER>> 
CLOVE VERB '<AIN>> 

O 

DICTIDNAPY-IN 
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Each entry has four items, a word. Its part of speech, the 
cognitive program and the generative program. 

The cognitive part Is a LISP s-expreseion (or program) 
that should evaluate to a number to indicate to LINGOL our 
satisfaction or otherwise with this choice of interpretation 
for this word. It Is relevant only when a given word has two 
dictionary entries corresponding to two parts of speech. 
Under these circumstances, we might write a program for each 
entry to Inspect the environment to see how reasonable the 
corresponding interpretation is. These programs would be 
executed if and when both interpretations were found to make 
sense given the context to the left, e.g., it would be 
executed in "the scout flies. .." but not in "the big files../', 
where "flies' 1 is listed as both a noun and a verb. This compo- 
nent of the entry need not concern us further here; we will 
remain neutral by writing everywhere, unless we happen to 
dislike the entry itself, in which case we will write -1, or 
-2 if we are In a bad mood. 

The generative part is a function destined to be tacked 
onto the surface structure. Since words are at the leaves 
of the tree, they have no arguments. In the case of "the", 
when the tree is evaluated, the corresponding leaf will return 
a list of one element (LE) as its value. The symbol is a 

quotation mark, and means "literally", so as LINGOL will not 
think (LE) Is a program to be executed. The other entries 
are all similarly structured. The reason we use a list of 
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one word rather than the word itself is that we are going 
to APPEND these lists together to form longer lists, 

Sow we want a grammar to make sense out of the words in 

combination. 

<GRAMHAR> 

^SENTENCE <NP PPED5 <REPLY (APPEND !L !P> CHflR>> * 
<NP <DET NP> (APPEND !L !R}> 

<MP NDUH ID) / 

<PRED (VERB MP> <flPPEMD !L !R>> 

O """ ; U 

GRRMMfiP-IM "'"■' *** 



Each rule la of the form (LEFT RIGHT COG GEN). The first 
two Items should be interpreted as a context-free rule 
LEFT -*- RIGHT, where RIGHT is either one category or a list 
of them if more are needed. At present LINGOL only permits 
RIGHT to have at most two categories; to get more, one should 
use extra names and rule? in the standard way. 

The item COG is exactly as for the corresponding 
dictionary item, except that it may be invoked for more complex 
types of ambiguity, usually structural. As with the dictionary, 
we shall write no non-trivial programs here, although we may 
occasionally use a negative number when we write a rule which 
we do not expect to need very often* 

The Item (EN is a more complex Item than its dictionary 
counterpart, since it can take arguments, which are written 
!D (down) if RIGHT is a syntactic category, and !L (left) or 
!R (right) if RIGHT is a list of two categories. These are 
not variables but programs which run the program for the 
corresponding subtree. 



The first rule takes the translation of the NP and the 
PHED and appends them into a single list. For example, if the 
NP were (LE CHIEN) and the PRED were {AIME LE HER), then 
(APPEND JL !R) would produce (LE CHIEN AIME LE MER). The 
function (REPLY L T) 13 a LINGOL function which allows the 
generative phase to type out on the console the words In L, 
followed by the value of T. The variable CHAR Is a LINGOL 
variable which makes available to the generative phase the 
character used to terminate the input string- (In the near 
future we shall give this to the cognitive phase instead, 
where it belongs-) In this case, we simply echo CHAR back to 
the console. 

The other rules are similar, but without the REPLY. 

Hopefully they are all self-explanatory. 

Let us try again to start LINGOL, 
* * *■■ 

< HELLO > 
• TYPE SENTENCE FOLLOWED BY . ! DP ? 

r - + - ■ 

1 ■'"■--- . r-j "'^ i -. < '** 



So far so good. Now for some sentences 

DOS LDVE SEfi- 
CHIEN RIM MER. 
542, MILLISECONDS. 

THE DOG LDVE THE SEfl. 
LE CHIEN RIM LE MER. 
391- MILLISECONDS. 

THE SEft LOVE DOG? 
LE MER RIM CHIEN? 
317. MILLISECONDS. 



There is nothing to say here except to comment on the 
timing. This includes reading in the sentence and performing 
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morphemic analysis (a feature to be described later), 

requiring about 30 milliseconds per word, or more If It is 

not in the dictionary* Parsing taVes f^on $0 to 100 msec ft 

per word depending on the complexity of the surface structure 

being produced, rather than on the size of the grammar. 

Parsing speed is essentially linear in the number of words 

in the Fimtence* given a reasonably intelligently written 

grammar of Englloh. The timing of the generative^ghase varies 

enormously, as a function of the complexity of the user's 

semantic programs. In these examples we are probably spending 

about 10 milliseconds per word. The slowness is due to LINGCt's 

being written in LISP. 

It would be nice tf we could inspect the tree on which 

we are operating. We can do this by telling LISP to set the 

flag TREE. To get LINQOL to pass on a message to LISP, 

precede it with a slash. 
I 



/<SET0 TREE T> 

T , 

THE DDG LOVE THE SEA. 




SENTENCE 

We have filled in the lines to show the connections.^ 
This device is one of several debugging aids. It is also 



o 
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possible to monitor the activity of the parser as It discover-; 
phrases, to see why it io not finding the right ones. The 
start and end positions of each discovered phrase are given, 
along with the rule used to discover It. 



/(SET© SHOWFDUHD T> 

T 

THE 

1 . 1 . DET THE 

DD6 

2. 2. NDUH DDG 

2. 2. NP NOUN 

1. 2. MP <DET . HP> 
LDVE 

3. 3. VERB LDVE 
THE 

4.4. DET THE 
SEft 

3. 5. NDUH SEft 
5. 5. HP NOUN 

4. 3. MP <DET . NP> 

3. 5. PPED (VERB . MP> 

1.5. SENTEHCE <MP . PPED> 

LE CHIEN AIM LE MER. 
63S. M I UL I SECONDS, 



We can also watch the EVAL mechanism for the semantics 
(called SEVAL) returning values up the tree, with the help 
of the LISP debugging aid TRACE: 
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•< TRACE (SEVAL VALUE » 
<SEVAL> 

THE DOG LOVE THE SEA. 



<1. 

<£. 
<3. 

<3. 
<3. 
<4. 
<4. 
<3. 
<2. 

ce. 

<3. 
<3. 
<3. 
<4. 
<4. 
<4. 
<5. 
C5. 
<4. 

<3. 
<£. 
<1. 
925. 

..-■ 



ENTER CEVAL) 
EMTER CEVAL) 
ENTER SEVAL > 
EXIT SEVAL CLE)) 
ENTER SEVAL > 
ENTER SEVAL) 

EXIT SEVAL «:chien>> 

EXIT SEVAL <XHIEN» 
EXIT SEVAL »;LE CHIEN)) 
ENTER SEVAL) 
ENTER SEVAL) 
EXIT SEVAL <flIM>> 
ENTER SEVAL) 
ENTEP SEVAL) 
EXIT SEVAL <LE)) 
ENTER SEVAL.) 
ENTER SEVAL) 
EXIT SEVAL <MER>> 
EXIT SEVAL <MER)) 
EXIT SEVAL <L£ HER)) 

EXIT SEVAL <AIM LE MEP>> LE CHIEN AIM LE NER 
EXIT SEVAL NIL) 
MILLISECONDS. 



The numbers indicate the depth In the parse tree 
(q.v. above). This routine is extremely helpful for verifying 
that all functions are producing the correct output, and also 
for discovering where in the tree SEVAL runs into trouble. 

We are not yet ready to translate the Canadian Hansard. 
Let us put in a variable to denote gender. The appropriate 
Bcope for the variable is an NP, since gender does not affect 
the verb. We need to tell LISP to change the grammar (our 
grammar 1b not yet elaborate enough to get LINGOL to do this 
for us). 
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•<GRAMMAP;> 

<NP <DET NP> <<LAMBDfi <GEND> <aPPEMD !L !R>> 'M> > 

GRAMMAR- IN 
LISP will now have replaced our old rule with the new 
one. (Only the components LEFT and RIGHT are used to identify 
and delete the old rule.) 

We have used LAM3DA rather than PROG to declare our new 
variable. Had we used PROG we would have said 

<PRDG XGEMD5 CSETQ GEMD 'H> (RETURN (APPEND !L !R>>> 



By using LAMBDA we save a SETQ and n RETURN. This is handy 

when there are a lot of variables to be SETQ'd. 

The scope of GEND is just the NP, i.e., those functions 

which are called directly or indirectly by IL and 'R here. 

GEND is set to "M" (masculine) as the default value (to enable 

us to eliminate specifying It in the dictionary), and will 

retain this value throughout its scope unless some function 

lower on the tree changes it, which we arrange now. 

'(DICTIDNAR'O . J 

<3EA MDUN CPRDG2 <SETQ GENU 'F> 'CMER>>> 

O 

BICTIQNARY-IN 



PR0G2 evaluates each of its arguments, but only returns 
the value of the second. 

We still have no way of using this information. Suppose 

we want determiner-noun agreement. 

•CDICTIDNAPY) 

(THE DET <CDR <flSSDC GEND '<<M LEXF LA>>>>> 

O 

DICTIONfiRY-IN 
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ASSOC is a LISP table-loo'njp function, and CDF. deletes 
the indicator In the discovered table entry (recall that we 
want (LE), not LE). 

Hopefully we will find that the sea is LA MER. 

THE DDG LDVE THE SEfi. 
LE CHIEN RIM LE MEP. . 
398. MILLISECONDS. 

There is a problem here with timing - we are trying to 
test GEND before it is set. The fault can be corrected from 
the HP rule, by doing !R before !L, on the grounds that the 
noun will never have to consult the determiner. This can be 
done by first assigning .'R to R. (For clarity, we revert to 

a PROG.) 

- , <eRRMMaR> 

<NP (DET NP> <PROG <GEND R> 

CSETO GEND 'H> 

<SETO R !R> 

(RETURN (APPEND !L R>>>> 

O 
GRAMMAR- IN 



* 

Now we can try again. 

THE DDG LDVE THE SEA. 
LE CHIEN AIM LA MER. 
436. MILLISECONDS. . 

THE SEA LDVE THE DOG . 
LA MER AIM LE CHIEN. 
436. MILLISECONDS. 



So It now seema to v:ork. Had both the DET and the NOUN 
depended on one other for various features. Instead of doing 
one before the other we would have ignored the order and done 
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the appropriate table lookup higher up In the tree - the 

dictionary would simply have passed the whole table up Instead 

of doing the lookup Itself. This would worV because the 

table lookup would be carried out with "complete Information", 

I.e., after both .'L and .'R had terminated execution. 

Verb conjugation seems to be next. 

•(GRAMMAR) 

(SENTENCE <NP PPED> 
((LAMBDA (PERSON ND> 

(REPLY (APPEND !L !R> CHAR)> 
3 'SING>> 
O 
GRAMMAR- IN 

■ * 

The default value for PERSON is 3, and for NO It is SING. 
To use these variables we need some dictionary entries. 



adictidnhry;> 

(love verb 

(list (count (cdr (assoc no 
■' '<(sing aime rimes aime> 

(plup aimdns aimez aim£nt>>>> 

PERSONS) 
(I NOUN (PPDG2 (SETQ PERSON 1> '<J£>>> 
(YOU NOUN (PROGS (SET© PERSON 2> '<TU>>> 

° ! 

DICTIONARY- IN ._ 

(COUNT L N) jlslds the Nth element of L. 
This now gives : 

THE DOG LOVE THE SEA. 
LE CHIEN AIME LA MER. 
443. MILLISECONDS. 

VDU LDVE THE SEA . 
TU AIMES LA MER. 
363. MILLISECONDS. 

It seems silly to have to write eo much In ft dictionary 
entry for a regular verb. Why not Just have a function REG 
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whlch adds the right ending to the stem? We will define It 
by using DSFUN. 



' 



/(DEFUN PEG (STEM) 

CLIST <CflT STEM -XDUNT <CDP <flSSOC ND 

'<<3IHG E ES E> 

<PLUR DHS EZ ENT>>>> 
PEPS0N>>>5 
REG 

(CAT concatenates bw:» firings.) 

Now we can -naVe most effective use of it. 



•<DICTlDNflRY> 

<LQVE VERB <REG 'flIM>> 

<HUNT VERB <PEG 'CHftS2>> 

<HIT VERB <REG 'FPflPP>> 

CCLIMB VERB <REG 'MDNT>> 

O 

BICTIQNRPY-IN 



We can still enter irregular verbs the old way, or better, 
we can define another function whose six arguments are the 
six conjugations. (We resist the temptation.) 

It would be nice to be able to distinguish singular 
and plural. This raises the morphological problem of detecting 
an "s" at the end of plurel words. One solution is to mp'-'e 
a dictionary entry for each plural word. But we can do better 
than this. LINOOL allowB the user to Identify suffixes by 
saying 

•CDEFPPDP S T SUFFIX) 

This says that it Is True that S is a SUFFIX. The same 
can be done for PREFIX. If LINOOL fails to find a word in 
the dictionary, it tries to remove a suffix. If it succeeds. 



X" 
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it looks In the dictionary for the stem. For as long as it 
keeps failing to find anything in the dictionary it keeps 
removing suffixes, and after that prefixes. When it is done 
the effect is as if the original word had been made several 
words, e.g., UNKINDNESSES becomes UN KIND NESS ES if UN is a 
prefix and NESS and ES are suffixes. All information that 
these words were once one word is discarded, which could 
conceivably create unwanted ambiguities, although it seems 
unlikely for most affixes. 

The word is eventually reassembled by the user's grammar, 
e.g.. 



'<GRFlMMflR> 

<NDUN <NOUN S> <PRDG£ <SETG ND 'PLUR> (LIST (CAT (CAR !L> 'S>> 

O 

GRAMHAR-IN . 

•(DICTIONARY) 
<S S 0) 

<> 

DICTIQNARY-IN 



THE DOGS LDVE THE SEA. 
LE CHIENS AIMENT Lfl MER . 
712. MILLISECONDS. 



Ah, the determiner rule is no longer valid. 

* * 

x(DICT1DNARY> 

(THE DET (COND ((EQ HD 'PLUR) '(LES>> 

<<CDR (ASSOC GEND '((M LEXF Lfl>>>>>>> 

O 

dictionapy-in 

the ddgs ldve the sea. 
les chiens aiment les nep. 
913. millisecond: . 
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AIbc we need a new PERSON and HO for the object, although 
GENDER 18 all right because It la In the NP rule. 

•(GRAMMAR) 

(PRED (.VERB NP) ■< APPEND !L « LAMBDA (PERSON NO) !R) 3 'SING)>> 

GRAMMAR- 1 M 

THE DDGS LOVE THE SEA. 
LES CHIEMS AIMENT LA MER. 
561. MILLISECONDS, 



The reason we keep finding errors is because we are writing 
the program as though we were beginners. With a little 
experience, the user can learn to anticipate most of these 
problems at the start. 

This scheme has the advantage that the user is not 

constrained to any one morphological system, but can write his 

own in the same language as he writes his semantics. It has 

another advantage in that morphological processing can be 

Interleaved with semantic processing. Por example, when LINGOL 

gives up on a word altogether, it assigns it the category 

UNKNOWN and supplies the word in the generative phase. If 

we want to implement Thome's (1963) closed-class dictionary, 

inwhlch unknown words are parsed as nouns, verbB or adjectives 

depending on which Interpretation makes the best syntactic 

sense, than we could write rules such as 

•(GRAMMAR) 

(NOUN UNKNDWN (LIST !D>) 
(VERB UNKNOWN <PEG !D)> 
: <> 

GRAMMAR-IN 



THE DDGS PREFER THE CATS. 
LES CHIENS PPEFERENT LES CATS 
737. MILLISECONDS. 
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Notice how the Issue of deciding what part of speech 
the word is is dealt with independently of, e.g., making 
"CAT" plural- Also notice that the parser correctly guessed 
the parts of speech, and went on to conjugate "correctly" the 
unknown verb- However, "cats" is a bit of an Anglicism, Our 
program is starting to look quite clever already without our 
having done very much to it yet. We have only seven grammar 
rules, one function (REG) and a few dictionary entries. 

In the example of Figure 5 (section 1), the rules involving 
UNKNOWN have for their generative component a program that 
queries the user about the translation. 

These examples could go on indefinitely. To see what can 
be achieved with a few more hours work, refer back to Figure 5. 
That example still has very little grammar - approximately 
twenty rules. However, it has a page of LISP functions for 
doing liason, various agreements, and handling tricky things 
like LES versus DES in the object position. 

These examples bring this section to an end. There is 
no section 3*3 on Pragmatics - this is entirely the user's 
problem. Figure 3 (section 1) gives examples from a LINGOL 
program in which the user successfully interfaced his 
semantics to quite non-trivial pragmatics. It is not yet 
clear whether LINGOL should ever address pragmatic issues. 
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4, Conclusions 

We have described a programming language for natural 
language processing programs. We discussed the reasons for each 
of the major design decisions- We presented a session Kith the 
system in which we developed a trivial fragment of an English- 
to-French translator- With adequate imagination, the reader 
should be able to project at least some of the potential of 
LINGOL. What may be more difficult to see are the present 
limitations of the system. 

We have already suggested that our separation of semantics 
from the syntax does not present serious problems. Whether 
this is true we leave to further experiments with LINQOL- It 
should be noted that LINGOL is still in its infancy; so far 
the author has invested approximately three months' work in it, 
over the two and a half years of its existence. 

At present, conjunction is not handled at all by LINGOL, 
except in so far as one may supply context-free rules for each 
syntactic category to be conjoined (which is most). This is 
tedious at best, and is not even always possible. One wants 
to deal not only with "The Chinese have short names and the 
Japanese long" but with "He eloped with and married the 
farmer's daughter. Neither of these are at all well handled 
by context-free grammars, regardless of what we write in the 
cognitive component of our rules- Winograd's system deals 
with these sorts of problems simply by being more procedure- 
oriented. This provides the necessary flexibility to deal 
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with pathological cases. 

Another difficult area is that of adverbs, which may 
appear in many places in a sentence, but which always modify 
the verb of the clause they appear in (unless they modify 
an adjective). It should not be necessary to give rules for 
each of the places an adverb may appear. It suffices to rely 
nainly on semantic connections to establish the role of the 
adverb, and this is one place where concept structures 
(Shank 1970) are of value. 

Both of these problems will be studied in the near future, 
to see how best to change LINCOL to deal with them without 
losing the attractive programming convenience afforded by 
context-free rules In conjunction with LISP semantics. In 
the meantime, the system as it stands at present is available 
from the author for experimental use. A LISP environment is 
required, with at least 20K words of memory. ; t An obvious 
application for LIN30L is as a pedagogical tool in a ccnputational 
linguistics course, for intrcducing students painlessly to one method 
of writing actual programs that do something useful with Erglish other 
than parsing it for the sake of the parse tree- We have used it fcr 
this purpose during the Independent Activities Period at MIT this 
January* One student wrote an EngUsh-toHanpolnted-Hebrew translator J : 
We ask only that users keep us up-to-date with the uses to which they 
put LLNOOL. 
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